docs(trtllm): correct the UCX env-var note and the EFA image facts on 1.4.0 - #13049
Merged
pvijayakrish merged 3 commits intoAug 12, 2026
Merged
Conversation
… 1.4.0 Adapted port of the main-side fix. The page lives at a different path on this branch, and the release-branch copy is staler than main in a way that changes the fix, so this is not a clean pick. Three corrections: The TRTLLM_USE_UCX_KVCACHE note stated the opposite of the actual behavior. Setting it with cache_transceiver_config.backend: DEFAULT does select direct UCX and is equivalent to backend: UCX; both produce a UcxConnectionManager. The EFA image was described as AMD64 only, with the claim that no pre-built ARM64 image is published. Both are false. The -efa tags are multi-arch manifests covering amd64 and arm64, confirmed against the registry, including 1.2.1-efa-amd64 despite its suffix. This matters here because the page told ARM64 readers the host mount was their only option. The ARM64 host mount breaks NIXL LIBFABRIC CUDA VRAM registration with fi_mr_reg failed: Bad address, asserting TensorRT-LLM and putting both workers into CrashLoopBackOff. It is kept documented with a warning scoped to that backend, and the pre-built image is now a real alternative to point at rather than one the page denied existed. Signed-off-by: Dan Gil <dagil@nvidia.com>
github-actions
Bot
added
docs
documentation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Comment on lines
+46
to
+49
| nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.3.0-efa | ||
| ``` | ||
|
|
||
| On 1.2.1 the same image is tagged `1.2.1-efa-amd64`. Despite the suffix that tag is a multi-arch manifest covering AMD64 and ARM64; the name was corrected to `-efa` in 1.3.0. Pull the tag exactly as written, since `1.2.1-efa` was never published. |
Contributor
There was a problem hiding this comment.
🔍 Recommended image tag lags the branch version and other docs still cite the AMD64-suffixed tag
The example tag is 1.3.0-efa, which matches the current stable inventory in docs/fern/reference/release-artifacts.mdx:96 even though this branch is versioned 1.4.0 (Cargo.toml:38). If a 1.4.0 EFA image is published for this release branch, this snippet will be stale on day one. Also, other pages still describe 1.2.1-efa-amd64 as an AMD64 image (docs/fern/kubernetes/cloud-providers/eks/eks.mdx:238, docs/fern/kubernetes/disagg-communication-guide.md:305), so the multi-arch correction made here is only partially applied across the docs set.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+65
to
+68
| > [!WARNING] | ||
| > Do not use this host mount with the LIBFABRIC backend on ARM64 / GB200. With `cache_transceiver_config.backend: NIXL` and `TRTLLM_NIXL_KVCACHE_BACKEND=LIBFABRIC`, mounting the host `/opt/amazon/efa` makes NIXL fail to register CUDA VRAM with `fi_mr_reg failed: Bad address`. TensorRT-LLM asserts immediately after, and both the prefill and decode workers enter `CrashLoopBackOff`. | ||
| > | ||
| > Use the pre-built `-efa` image instead. Removing the mount and relying on the EFA SDK shipped in that image makes an otherwise identical deployment serve inference on the same pair of nodes. |
Contributor
There was a problem hiding this comment.
🔍 New warning references a backend value not otherwise documented on this page
The warning uses cache_transceiver_config.backend: NIXL, but the rest of the page only documents DEFAULT and UCX for that key, with the NIXL sub-backend selected via TRTLLM_NIXL_KVCACHE_BACKEND. Readers may be unsure whether NIXL is a valid literal for cache_transceiver_config.backend; a short clarification (or using DEFAULT if that is what was actually tested) would avoid ambiguity.
Was this helpful? React with 👍 or 👎 to provide feedback.
Contributor
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
Collaborator
Author
|
/ok to test 7d0805e |
Replaces the hand-written NOTE in this cherry-pick with the table Chi Xing landed on main in #13015, so the release branch matches main rather than carrying a second, differently-worded correction of the same defect. Both fix the inversion that 6590511 reports. His version is the better one: it documents all four TRTLLM_USE_*_KVCACHE variables and their precedence rather than only the UCX one, states the not-set case, and cites CacheTransceiverConfig._resolve_default_backend so the claim can be rechecked against a specific TensorRT-LLM version. Kept from the earlier revision: the sentence naming which transceiver class each path logs. That is the part QA needed to tell the two apart from a worker startup log, and #13015 does not carry it. The file lives at a different path on this branch, so this is an adapted port rather than a clean pick. Signed-off-by: Dan Gil <dagil@nvidia.com>
pvijayakrish
approved these changes
Aug 12, 2026
dagil-nvidia
added a commit
that referenced
this pull request
Aug 14, 2026
Re-snapshot docs/ from main (46 further commits since the first snapshot), re-apply the release ledger at the new paths - the 1.4.0 image-tag pins, the shipped-behavior reference sync (#12983), the CKF metric caveats (#13149), and the UCX/EFA facts (#13049) - and sync docs CI plus both READMEs from main so no link targets the deleted old layout. Promote v1.4.0 to current release across the docs values: releases.data.ts (CURRENT_* consts, v1.4.0 release entry with shipped pins SGLang 0.5.16 / TRT-LLM 1.3.0rc22 / vLLM 0.26.0 / NIXL 1.3.0-1.3.2 / UCX 1.21.x, CUDA rows, artifact tags) with regenerated reference pages, and bump copy-paste container tags in install guides (EN/zh-CN), READMEs, profiler and guide examples. Recipe-mirroring pages keep their deliberate validation pins. Signed-off-by: Dan Gil <dagil@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adapted port of #13048. The page sits at a different path on this branch, and the release-branch copy is staler than main in a way that changes the fix, so this is not a clean pick.
1. The
TRTLLM_USE_UCX_KVCACHEnote said the opposite of the truthThree deployments differing only in that variable, transceiver class read from each startup log:
DEFAULTNixlTransferAgent ... using NIXL backend: UCXDEFAULTTRTLLM_USE_UCX_KVCACHE=1UcxConnectionManagerUCXUcxConnectionManagerB and C are the same, so the variable does select direct UCX.
2. The EFA image facts were wrong, and that is load-bearing here
This branch says the pre-built EFA image is AMD64 only and that no pre-built ARM64 image is published. Both are false. Verified against the registry:
1.3.1-efa1.3.0-efa1.2.1-efa-amd641.2.1-efa-amd64is multi-arch despite its suffix. This is not a cosmetic correction: the page told ARM64 readers the host mount was their only option, so the warning below would otherwise leave them with no path at all. Main already carries the corrected text and this brings the branch in line with it.3. The ARM64 host mount breaks LIBFABRIC
With
backend: NIXLandTRTLLM_NIXL_KVCACHE_BACKEND=LIBFABRIC, mounting the host/opt/amazon/efamakes NIXL fail to register CUDA VRAM withfi_mr_reg failed: Bad address. TensorRT-LLM asserts immediately after and both workers enterCrashLoopBackOff. Removing the mount and using the SDK in the pre-built-efaimage makes the identical deployment serve on the identical nodes.The mount stays documented with a warning scoped to that backend, rather than deleted, since it is still how you keep the SDK in step with the host driver.
Notes
Docs only. The relative link to Release Artifacts was repointed to this branch layout and the target exists. This converges the page toward main, which is the direction #12950 takes the tree anyway.